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Abstract 

Consider throwing n balls at random into m urns, each ball land- 
ing in urn i with probability pi. Let S be the resulting number of 
singletons, i.e., urns containing just one ball. We give an error bound 
for the Kolmogorov distance from S to the normal, and estimates on 
its variance. These show that if n, m and (pi, 1 < i < m) vary in 
such a way that sup^pj = 0(n _1 ), then S satisfies a CLT if and only 
if n 2 J2i Pi tends to infinity, and demonstrate an optimal rate of con- 
vergence in the CLT in this case. In the uniform case (pi = m^ 1 ) 
with m and n growing proportionately, we provide bounds with bet- 
ter asymptotic constants. The proof of the error bounds are based on 
Stein's method via size-biased couplings. 

1 Introduction 



Consider the classical occupancy scheme, in which each of n balls is placed 
independently at random in one of m urns, with probability Pi of going into 
the ith urn (pi + p 2 + ■ ■ ■ + p m — 1). If ^ denotes the number of balls 
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placed in the zth urn, then (N\, . . . ,N m ) has the multinomial distribution 
Mult(n;pi,p2; • • • ,Pm)- A special case of interest is the so-called uniform case 
where all the pi are equal to 1/m. 

A much-studied quantity is the number of occupied urns, i.e. the sum 
J2i l{-^Vj > 0}. This quantity, scaled and centred, is known to be asymptoti- 
cally normal as n — > oo in the uniform case with mccn, and a Berry-Esseen 
bound for the discrepancy from the normal, tending to zero at the optimum 
rate, was obtained for the uniform case by Englund [4], and for the general 
(nonuniform) case, with a less explicit error bound, by Quine and Robin- 
son [12] . More recently, Hwang and Janson [9] have obtained a local limit 
theorem. A variety of applications are mentioned in [5] ('coupon collector's 
problem, species trapping, birthday paradox, polynomial factorization, sta- 
tistical linguistics, memory allocation, statistical physics, hashing schemes 
and so on'). Also noteworthy are the monographs by Johnson and Kotz [10] 
and by Kolchin et al. [UJ; the latter is mainly concerned with models of 
this type, giving results for a variety of limiting regimes for the growth of 
m with n (in the uniform case) and also in some of the non-uniform cases. 
There has also been recent interest in the case of infinitely many urns with 
the probabilities pi independent of n [HE]. 

In this paper we consider the number of isolated balls, that is, the sum 

l{Ni = 1}. This quantity seems just as natural an object of study as 
the number of occupied urns, if one thinks of the model in terms of the balls 
rather than in terms of the urns. For example, in the well-known birthday 
paradox, this quantity represents the number of individuals in the group who 
have a unique birthday. 

In the uniform case, we obtain an explicit Berry-Esseen bound for the 
discrepancy of the number of isolated balls from the normal, tending to zero 
at the optimum rate when m cx n. In the non-uniform case we obtain a 
similar result with a larger constant, also finding upper and lower bounds 
which show that the variance of the number of isolated balls is B(n 2 YliPf)- 
The proof of these bounds, in Section [5j is based on martingale difference 
techniques and somewhat separate from the other arguments in the paper. 

Our Berry-Esseen results for the number of isolated balls are analogous 
to the main results of [I] (in the uniform case) and [12] (in the non-uniform 
case) for the number of occupied urns. Our proofs, however, are entirely 
different. We adapt a method used recently by Goldstein and Penrose [8] for 
a problem in stochastic geometry (Theorem 2.1 of [8]). 

Our method does not involve either characteristic functions, or first Pois- 
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sonizing the total number of balls; in this, it differs from most of the ap- 
proaches to problems of this type adopted in the past. As remarked in [H] 
'almost all previous approaches rely, explicitly or implicitly, on the widely 
used Poissonization technique', and this remark also applies to [9] itself. One 
exception is Chatterjee [3], who uses a method not involving Poissonization 
to give an error bound with the optimal rate of decay (with unspecified con- 
stant) for the Kantorovich-Wasserstein distance (rather than the Kolmogorov 
distance, as here) between the distribution of the number of occupied urns 
and the normal, in the uniform case. 

We believe that our approach can be adapted to the number of urns 
containing k balls, for arbitrary fixed k, but these might require significant 
amounts of extra work, so we restrict ourselves here to the case with k = 1. 

Our approach is based on size-biased couplings. Given a nonnegative 
random variable W with finite mean = EW, we say W has the W size 
biased distribution if P[W G dw] = (w/fj,)P[W G dw), or more formally, if 

E [^/(VF)] = fjMf(W') for bounded continuous functions /. (1.1) 

Lemma 13.11 below, due to Goldstein [7] , tells us that if one can find coupled 
realizations of W and W which are in some sense close, then one may be 
able to find a good Berry-Esseen bound for W . It turns out that this can be 
done for the number of non-isolated balls. 

2 Results 

Let n G N and m = m(n) G N with m > 4. Let = (p^\ 1 < x < m) 
be a probability mass function on [m] := {1,2,..., m}, with pi n) > for all 
x G [m]. Let X and X^, 1 < i < n be independent and identically distributed 
random variables with probability mass function p = (we shall often 
suppress the superscript (n)). Define Y = Y(n) by 



In terms of the urn scheme described in Section [TJ the probability of landing 
in Urn x is p x for each ball, Xj represents the location of the ith ball, Mj 
represents the number of other balls located in the same urn as the ith ball, 
and Y represents the number of non-isolated balls, where a ball is said to 



n 



n 
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be isolated if no other ball is placed in the same urn as it is. Thus n — Y 
is the number of isolated balls, or in other words, the number of urns which 
contain a single ball. 

Let Z denote a standard normal random variable, and let <&(£) := P\Z < 
t] = (27r) -1 / 2 exp(— x 2 /2)dx. Given any random variable W with finite 
mean \i w and standard deviation aw satisfying < aw < °o, define 



D w '■= sup 



P 



W - nw < t 
a w 



the so-called Kolmogorov distance between the distribution of W and the 
normal. We are concerned with estimating Dy. 

We refer to the case where p x = m _1 for each x G [m] as the uniform 
case. Our main result for the uniform case provides a normal approximation 
error bound for Y, which is explicit modulo computation of \iy and ay, and 
goes as follows. 

Theorem 2.1. In the uniform case, if a Y > 24fiy, then 

Dy <°A + (!»)(™ + V +23 ^(^) (2.2) 
oy \a Y ) \ a Y ap J 

with 

16 4 24 / n n\ 

V{n, m := - + + — 2 + + - 2.3 

n n{n — 1) m \ m — 3 m J 

For asymptotics in the uniform case, we allow m = m(n) to vary with n. 
We concentrate on the case where m = G(n). In this case both fiy and a\ 
turn out to be 0(n) as n — > oo, and thus Theorem 12.11 implies Dy is 0(n -1 / 2 ) 
in this regime. More formally, we have the following. 

Theorem 2.2. Suppose n,m both go to infinity in a linked manner, in such 
a way that n/m — > a E (0, oo) . Then with g(a) := [er a — e~ 2a (a 2 — ct + l)) 1 / 2 , 
we have in the uniform case that g(a) > and 

Q g / 1 g— a \ 

lim sup n 1/2 D Y < — — + 256 . r 
n-oo g[a) V 9{®r ) 

+92(^-^)(l + 3a(l + a)) 1 / 2 . (2.4) 
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In the case a = 1, the right hand side of (12.41) . rounded up to the nearest 
integer, comes to 2236. Theorems 12. II and 12.21 are proved in Section HI 

We now state our results for the general (non-uniform) case. Given n we 
define the parameters 

||p|| := sup (p x ); 7 = 7M := max(n||p||, 1). (2.5) 

x£[m] 

For the large- n asymptotics we essentially assume that 7(n) remains bounded, 
or at least grows only slowly with n; see Corollary 12.11 below. First we give 
a non-asymptotic result. 

Theorem 2.3. It is the case that 

D Y > min (l/6, (Sne^^a- 1 ) , (2.6) 



and if 



and also 



then 



with 



IIpII < 1/11 (2-7) 

n > 837 2 (l + 37 + 37 2 )e L057 , (2.8) 

D Y < 81657 2 e 2 ' l7 (577 + 23C(^))a Y \ (2.9) 

C(i) := 10(82 7 7 + 82 7 6 + 8O7 5 + 47 7 4 + 12 7 3 + 12 7 2 ) 1/2 . (2.10) 



It is of use in proving Theorem I2.3[ and also of independent interest, to 
estimate the variance a Y in terms of the original parameters (p x ,x G [m]), 
and our next result does this. Throughout, we write J2 X f° r Y^=r 

Theorem 2.4. It is the case that 

Vary<8n 2 ^p 2 , (2.11) 

X 

and if p~7[ ) and fl^.gp hold, then 

VarF > (7776)" 1 7 _2 e~ 2 - 1 V^p 2 . (2.12) 
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If 7(71) remains bounded, i.e. sup n 7(n) < 00, then both (j2.7p and (12.81) 
hold for large enough n. Hence, the following asymptotic result is immediate 
from Theorems 12.31 and 12.41 

Corollary 2.1. Suppose sup n 7(n) < 00. Then the following three conditions 
are equivalent: 

• n2 XL pI ~ > 00 as n ~^ 00 / 

• cry — > 00 as n — > 00 ; 

• (Y — KY)/o~y converges in distribution to Z as n — > 00. 
// these conditions hold, then 



Remarks. In the uniform case, Theorem [272] provides an alternative proof 
of the central limit theorem for Y when m = 0(n) (see Theorem II. 2. 4 on 
page 59 of with error bounds converging to zero at the optimum rate. 
Corollary 12.11 shows that in the uniform case, if n 2 /m — ► 00 and n/m remains 
bounded, then Dy = 0((n 2 /m)~ 1//2 ). Corollary 12.11 overlaps Theorem III. 5. 2 
on page 147 of [llj but is under weaker conditions than those in [11], and 
provides error bounds not given in [11]. 

The condition that 7(71) remain bounded, in Corollary 12. 11 is also required 
by [12] for the analogous Berry-Esseen type result for the number of occupied 
boxes, though not by [9J for the local limit theorem for that quantity. In 
(12.71) . which is used for the non-asymptotic bounds, the bound of jj could 
be replaced by any constant less than | without changing anything except 
the constants in (ETJ) and (l2~T2l . 

As always (see remarks in [9], [33], pQ), it might be possible to obtain 
similar results to those presented here by other methods. However, to do so 
appears to be a non-trivial task. In [11] the count of the number of isolated 
balls is treated separately, and differently, from the count of occupied urns 
or the count of urns with k balls, k = or k > 2. Poisson approximation 
methods might be of use in some limiting regimes (see [2], Chapter 6), but 
not when the ratio between E \Y] and Var[Y] remains bounded but is not 
asymptotically 1, which is typically the case here. 



D Y = e(a Y 1 ) = e[ (u 2 J2pI 




) 



1/2 



) 
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Exact formulae can be written down for the probability mass function 
and cumulative distribution function of Y. For example, using (5.1) on page 
99 of [5] , the cumulative distribution of Y may be written as 

m f ' — 1 \ 

P[Y <n-k] = P[n-Y>k] = $^(-l) i ~ fc U JiSj 

j=k \ ~ / 

with Sj a sum of probabilities that j of the urns contain one ball each, i.e. 

a;i<^2<-"<a;j<'Ti \ i=l 

We shall not use this formula in obtaining our normal approximation results. 




3 Lemmas 

A key tool in our proofs is the following result, which is a special case of 
Theorem 1.2 of [7], and is proved there via Stein's method. 

Lemma 3.1. |7|/ Let W > be a random variable with mean fi and variance 
a 2 G (0, oo), and let IV s be defined on the same space, with the W -size biased 
distribution. If \W S — W\ < B for some B < cr 3 / 2 / v /6/I, then 



where 



OAB u /64B 2 4B 3 . 

D w < + 4 + ^ + 23A 3.1 

o~ o~ V o~ a 1 



A := y/Vax(E(W*-W\W)). (3.2) 



Our next lemma is concerned with the construction of variables with 
size-biased distributions. 

Lemma 3.2. Suppose W is a sum of exchangeable indicator variables £i, . . . , £ n , 
with P[W > 0] > 0. Suppose . . . , £' n are variables with joint distribution 

£(^,...,o = >c(6,... J ^iei = i). 

Then the variable W = Y^7=i C ^ as ^ e ^ s ^ ze biased distribution. 



7 



Proof. See Lemma 3.3 of [8]. □ 

Let Bin(n,p) denote the binomial distribution with parameters n G N 
and p G (0, 1). The following lemma will be used for constructing the desired 
close coupling of our variable of interest Y, and its size biased version Y', so 
as to be able to use Lemma 13.11 

Lemma 3.3. Let v G N and p G (0, 1). Suppose N ~ Bm(v,p). Define 
fork = 0,1,2,...,!/, by 

( P[N>k\N>0]-P[N>k] • , n < , < _ 1 
7T fc := < P[tf=fc](l-(fc/«>)) V u V K ^ Z/ 1 

I if k = u. 

Then < n k < 1 for each k G {0, 1, . . . , u}. 

Proof. See Lemma 3.5 of [8]. □ 

Our next lemma is a bound on correlations between variables associated 
with different balls in the urn model. Recall the definition of at (12. ip 



Lemma 3.4. Let k G N wii/i 2 < k < n, and suppose that for i — 1, . . . , k, 

ipi is a real-valued function defined on {0} U [n — 1], with E[-0i(Mi)] = 0, 
and set \\ipi\\ := sup ie[n] {\ipi(£ - 1)|} and rng(^) := sup i£[n] {ipi(£ - 1)} - 
inffc e [ n ]{'0j(£ — 1)}. Then in the uniform case, 



E 



i=i 



s ^(ni*i)"«(«( j +fe)+s) 



Proof. SetW := Ui=i M M i)- Write xW for 0*1, • • • , **) and x for k- 
vectors x = (xi, . . . , X&) of possible values of X( fc ). Let F be the set of vectors 
x = (xi, . . . , Xk) such that X\ ^ Xj for j = 2, . . . , fc, so that {X^ G F} is the 
event that each of Balls 2, . . . , k lands in a different urn from Ball 1. Then 
P[X( fc ) g F c ] < (k - l)/m, and |W| < fti IHM, so that 



IEWI 



£[jy|X (fe) G F c ]P[X (fc) G F c ] + J^P[X (fc) = x]E [W\X {h) = x] 

xS-F 
fc 

< ((A; - l)/m) JJ HViH + sup |E [W|X^ = x]|(3.4) 



i=i xeF 
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Fix x G F. We group the urns into three 'boxes'. Let Box 1 consist of the 
urn containing Ball 1, and let Box 2 be the union of the urns containing 
Balls 2,3, ... ,k; this could be the union of any number up to k — 1 of urns 
depending on how many of x 2 ,x 3 , ■ ■ ■ ,Xk are distinct, but since we assume 
x G F does not overlap Box 1. Let Box 3 consist of all other urns except 
those in Box 1 or Box 2. For i — 1, 2, 3, let N be the number of balls in Box 
i, other than Balls 1, . . . , k. Let h{k) be the expected value of Yli =2 ipi(Mi), 
given = x and given that N 2 = k. Then 

E [W\X. {k) = x] = E hMiVi)/^)]. (3.5) 

Also, given X( fc ) = x, (iVj)f =1 have the multinomial distribution 

(JV ls N 2 , N 3 ) ~ Mult (n 1 - — ) , (3.6) 

\ mm m J 

and a denotes the number of distinct values taken by x 2 , ■ ■ ■ , X}.. 

We give a coupling of Ni to another random variable iV] with the same 
distribution as N± that is independent of A^2, for which we can give a useful 
bound on P[Ni ^ N[). 

Consider throwing a series of coloured balls so each ball can land in one of 
the three boxes, where the probabilities of landing in Boxes 1,2,3 are 1/m, 
a/m, {m — a — 1)/ m respectively. First, throw n — k white balls and let 
Ni,N 2 ,N^ be the number of white balls in Boxes 1,2,3 respectively. Then 
pick out the balls in Boxes 1 and 3, paint them red, and throw them again. 
Then throw enough green balls so the total number of green and red balls is 
Ti—1. Finally take the red balls in Box 2 (of which there are of iVo, say), 
paint them blue, and throw them again but condition them to land in Boxes 
1 and 3 (or equivalently, throw each blue ball again and again until it avoids 
Box 2). Then (with obvious notation, superscripts denoting colours) set 

JVi = Nl + Nl N 3 = iV 3 r + Nl N{ = iV{ + Nf. 

Then (Ni,N 2 ,N 3 ) have the multinomial distribution given by (13.61) . Also, 
N[ has the same Bin(n — 1, — ) as Mi in the statement of the lemma, and N[ 
is independent of N 2 . Since iV( = Ni - + Nf, we have that 

P[JVi ^ N[] < E [JVf] + E [N§ 

<-(k-l + EN 2 )+ ( [ /m ) E [N ] 
m \ 1 — (k — lj/m J 

< — (k — l + {k— l)n/m + ( 1 - - ) (k - l)n/m ] 

m \ \l — [k — l)/mj J 



9 



so that 



|E[(^(iVi) -MN[))h(N 2 )]\ < Pim ± ^mg^i)!! 

i=2 
4 



^ k — 1 ^m + n | 



r?) 



m — + 1 



i=2 



and since N[ is independent of N 2 with the same distribution as Mi, so that 
E-0i(iV{) = by assumption, 



E[^(iV0M^2)] = 0, 



so by (J33D, 



ie [w| X w=x]i<*^rr^i + 



m 



n 



m — + 1 



rng 



i=2 



Combined with (13.41) . and the fact that ||'0i|| < rng(^i) since E [-01 (Mi)] = 0, 
this demonstrates the result. □ 



Next, we adapt Lemma [3.41 to the non- uniform setting. In this case, we 
need to allow ipi to depend on the location as well as the occupation number 
associated with the ith ball. Consequently, some modification of the proof is 
required, and the constants in Lemma 13.41 are better than those which would 
be obtained by simply applying the next lemma to the uniform case. 

Lemma 3.5. Suppose that for i = 1,2,3,4, ipi is a real-valued function 
defined on [m] x {0,1,..., n — 1}, with E [-01 (Xl, Mi)] = 0, set \\ipi\\ : = 
^P(x,k)e[mMn]{\M^,k - 1)|} and set rng(^) := sup (l . k)e[m]x[n] {\Mx, k ~ 
1)|} -inf( a . jfc ) e[m ,] X [ n ]{|V'i(s,A; - 1)|} . Then 



|E [MXi,M 1 )^ 2 (X 2 ,M 2 )]\ < (3 + 3 7 )rng(^ 



E 



p. 



(3.7) 



and 



E 



|I'.v(.V,,.U, 



i=i 



< 



(9 + 9 7 )rng(^i) mil^ll 



,i=2 



(3.8) 
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Proof. We first prove (13.81) . Throw n balls according to the distribution 
p, with four of them distinguished as Ball 1, Ball 2, Ball 3 and Ball 4. For 
i — 2, 3, 4, let Zi be the location of Ball i and let Ni be the number of other 
balls in the same urn as Ball i. Set A = Uf =2 {Zi}, the union of the locations 
of Balls 2,3, and 4. 

Now suppose the balls in A are painted white. Let the balls not in A 
(including Ball 1 if it is not in A) be re-thrown (again, according to the 
distribution p). Those which land in A when re-thrown are painted yellow, 
and the others are painted red. 

Now introduce one green ball for each white ball, and if Ball 1 is white, 
let one of the green balls be labelled Ball Gl. Throw the green balls using 
the same distribution p. Also, introduce a number of blue balls equal to the 
number of yellow balls, and if Ball 1 is yellow then label one of the blue balls 
as Ball Bl. Throw the blue balls, but condition them to avoid A; that is, use 
the probability mass function (p x /({ — J2 y eAPy)' x e l m ] \ A) f° r the ^ me 
balls. 

Set Zi to be the location of Ball 1 (if it is white or red) or Ball Bl (if 
Ball 1 is yellow). Set Z[ to be the location of Ball 1, if it is red or yellow, or 
the location of Ball Gl (if Ball 1 is white). Let iVf , N{, and Nf respectively 
denote the number of white, red, and blue balls at location Z%, not counting 
Ball 1 or Ball Bl itself. Let Nf, Nf, and Nf respectively denote the number 
of yellow, red, and green balls at location Z[, not counting Ball 1 or Ball Gl 
itself. Set 

JVi = iVJ° + N[ + Nf, N[ = Nf + Nf + Nf. 

Then ((Zi,Ni)f =1 ) have the same joint distribution as ((Xj, Mj)f =1 ). Also, 
(Z[,N[) has the same distribution as (Xx,Mi) and (Z[,N[) is independent 
of ({Zi, Ni)f =2 ). Finally, if Ball 1 is red then Z x = Z[ and N[ = N t -Nf + Nf, 
so that 

P[(Z U N ± ) ^ (Z[, N[)] < E [Nf] + E [Nf] + 2P[Z[ e A]. (3.9) 

Now, 

4 

P[z[ e A] < Y, p iXi = x<] = 3$>*- ( 3 - 10 ) 

i=2 x 

Also, if N g denotes the number of green balls, not including Ball Gl if Ball 
1 is green, then by (12. 5p . 

E[N g ] < 3 + 3n||p|| < 3(1 + 7) 
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and also E [Nf\N g ] < N g Y, x vl, so that 



E [Nf] = E [E lN?\N g }} < 3(1 + 7) ]T 



pl- 



(3.11) 



If iVy denotes the number of yellow balls, other than Ball 1, then by ( 12.51) . 



E[N y ] < 3n||p|| < 37 



(3.12) 



and by 



so that 



(3.13) 



E[iV 1 6 ]=E[E[iV 1 b |iV y ]]<67^^. 

X 

Set W := Yt =2 ^i{Z h Ni). By ([S3D, (JSZHD, <&H§ and ([313]), 

4 

lEfW^iC^iVO-Vi^iVO)]! <P[{Z 1 ,N l )^{Z' l ,N' 1 )]xng^ 1 )\{U i \\ 

i=2 

< (9 + 97)rng(^) (f[ ||^|| ) J^. (3.14) 



v i=2 



Since (Z(,iV{) is independent of ((Zi, Ni)j =2 ) with the same distribution as 
(Xi, Mi), and E Vi(Xi, Mi) = by assumption, E [Wip^, N[)] = 0. Hence, 



E 



E [WMZi, N,)] = E [W^(^i(Zi, Nx) - MZ[, N[))}, 



and then (13. 8p follows by (I3.14p . The proof of (13.71) is similar, with the factors 
of 3 replaced by 1 in ([XTO]) . (I3TTT1) and (I3TT2|1 . □ 



4 Proof of Theorems 12.11 and 12.2 



Proof of Theorem \2.1[ Recall the definition ( 12. II) of Mj and K. Assume the 
uniform case, i.e. assume p = {m~ x , m _1 , . . . , m~ l ). Let ^ := l{Mj > 0} 
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be the indicator of the event that ball i is not isolated. Then Y = X^=i£i> 
and since are exchangeable, a random variable Y' with the size-biased 
distribution of Y can be obtained as follows; see Lemma 13.21 Let / be a 
discrete uniform random variable over [n], independent of X%, . . . , X n . Given 
the value of /, let X' = (X[, . . . ,X' n ) e [m] n be a random n-vector with 
£{X' 1 ,...,X' n )=C(X 1 ,...,X n \Z J = l). Set 

n 

y' '= E UUieHUi}^' = X!}}. 
i=i 

To apply Lemma [3.11 we need to find a random variable Y", coupled to Y, 
such that C(Y") = C(Y') and for some constant B we have \Y' — Y\ < B 
(almost surely). To check C(Y") = C(Y), we shall use the fact that if M\ 
denotes the number of entries X'- of X' that are equal to Xj, other than X\ 
itself, then (i) given / and Xi, Mj has the distribution of a Bin(n — 1, 1/m) 
variable conditioned to take a non-zero value, and (ii) given /, X\ and Mj, 
the distribution of X' is uniform over all possibilities consistent with the 
given values of /, X' T and Mj. 

Define the random n- vector X := {X\ . . . ,X n ). We can manufacture a 
random vector X" = (X", . . . ,X%), coupled to X and (we assert) with the 
same distribution as X', as follows. 

• Sample the random variables (Xi, . . . , X n ). Define Mj by (12. ip . 

• Sample a value of / from the discrete uniform distribution on [n], in- 
dependent of X. 

• Sample a Bernoulli random variable B with P[B = 1] = itMii where 
(7Tfc, k > 0) is given by (13.31) with v = n — 1 and p = mT x . (By Lemma 

E31 o < 7r fc < l.) ' 

• Sample a value of J from the discrete uniform distribution on [n] \ {I}. 

• Define (X?,...,X'£ by 

_ ( Xj if % = J and B = 1 
1 \ Xi otherwise. 

Thus X" is obtained from X by changing a randomly selected entry of X 
to the value of Xj, if B — 1, and leaving X unchanged if B = 0. 
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We claim that £(X") = £(X'). To see this define N := M u and set 
N" ■= -l+EILi 1{X? = X'j'}. Then N has the Bin(n-1, m" 1 ) distribution, 
while N" always takes the value either N or N+ 1, taking the latter value in 
the case where B = 1 and also Xj ^ Xj. Thus for any k G {0, 1, . . . , n — 1}, 

P[jV" > fc] = P[iV > k] + P[JV = A;]7r fc (l - (k/(n - 1))), (4.1) 

so by the definition of n k , £(N") = £{N\N > 0). This also applies to 
the conditional distribution of iV" given the values of / and Xj. 

Given the values of N", I and X", the conditional distribution of X" 
is uniform over all possibilities consistent with these given values. Hence, 
£(X") = £(X'). Therefore setting 

n 

F":=^l{U, eWUl} {X; = Xr}}, 
i=i 

we have that £(Y") = £(Y'), i.e. Y" has the size-biased distribution of Y . 

The definition of X" in terms of X ensures that we always have \Y — Y"\ < 
2 (with equality if Mi = Mj = 0) ; this is explained further in the course 
of the proof of Proposition 14.11 below. Thus we may apply Lemma 13.11 with 
B = 2. Theorem 12.11 follows from that result, along with the following: 

Proposition 4.1. It is the case that Var(E [Y" — Y\Y}) < i](n,m), where 
t](n,m) is given by (\2.3\i . 

Proof. Let Q be the cr-algebra generated by X. Then Y is ^-measurable. 
By the conditional variance formula, as in e.g. the proof of Theorem 2.1 of 
0, 

Var(E \Y" - Y\Y\) < Var(E \Y" - Y\Q\), (4.2) 

so it suffices to prove that 

Var(E [Y" - Y\Q}) < r](n, m). (4.3) 

For 1 < % < n, let Vi denote the conditional probability that 3=1, given 
X and given that I = i, i.e. 

K = ^ Mi ■ 
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Let Rij denote the increment in the number of non-isolated balls when the 
value of Xj is changed to X^. Then 

E[Y"-Y\G]= 1 V ViRij 
n(n — 1) ^ 

where YlajVi^j denotes summation over pairs of distinct integers i,j in [n\. 
For 1 < i < n and j ^ i, let 

Si := 1{M, = 0}; Tj := 1{M, = 0} - 1{M, = 1}; 

Qij : = 1{M< = l}l{Xi = Xj}. 

Then we assert that Rij, the increment in the number of non-isolated balls 
when ball j is moved to the location of ball i, is given by Rij := Si + Tj + Q^. 
Indeed, if Xj ^ Xj then is the increment (if any) due to ball % becoming 
non-isolated, while Tj is the increment (if any) due either to ball j becoming 
non-isolated, or to another ball at the original location of ball j becoming 
isolated when ball j is moved to the location of ball i. The definition of Qij 
ensures that if Xi = Xj then Si + Tj + Qij = 0. Thus, 

E[Y"-Y\g} = — L_- £ Vi(Si + T j + Qij) 



1 n i 

n ^— ' nin — 1) 

where we set 

r, := Si + ( — ^- J Qij = HMi = 0} + ( — ^-J 1{M< = !}• (4-5) 
Put a := E [Vi] (this expectation does not depend on i). Then by (I4.4p . 
E[Y"-Y\g]= 1 -±(V t r l + aT l )+ £ I^ffi 

Since (x + y) 2 < 2{x 2 + y 2 ) for any real x, y, it follows that 
Var (E [Y' — Y\Q)) < 2Var U- (Vm + aTM 

+2Var V (4.6) 
^ n(n-l) 
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From the definitions, the following inequalities hold almost surely: 

- 1 < Ti < 1; < Vi < 1; < r< < 1; (4.7) 

and hence 

- 1 < Vi - a < 1; - 1 < (V - a)Tj < 1; - 1 < + aT; < 2(4.8) 

Set Z; := V^r< + aT { ,_ and Z, := ^ - EZj. By g]3]), VarZi < EZf < 4. Also 
by (US), we have |Z;| < 3, and - 1 ZZ, < Z, < 2 - EZ;. Hence by the 
case k = 2 of Lemma 13.41 

Cov(Z 1; Z 2 ) = E [2,2,] <-(l ( " ^ 1 " " "' 



m \ — 1/ \ m 

Thus for the first term in the right hand side of (14.61) . we have 



Var 



I E ^) = ^~ 1 Var(Z 1 ) + CovCZx, Z 2 ) 



4 9 / f n \ fn + m\\ 

<- + -!+ 7 + • 4.9 

n m \ \m — 1 / \ m // 

For the second term in the right hand side of (j4.6j) . set := — a. By (J47TJ) , 
— a < V < 1 — a, and |Tj| < 1. Hence by the case k = 4 of Lemma [3.41 

Cov(ViT 2 , V 3 T 4 ) < E [VxT^Tj < - ( 2 + ( — ^ J + - 

m \ \m — 3 / m 

By (14.81) . we can always bound Cov (ViTj, Vi'Tji) by 1. Hence, expanding 
Var^V.x V^Ij in the same manner as with (I6.25|) below, yields 

^ 3 fo , f n \ n 



Var V -^-<- 2 



n(n — 1) m \ \m — 3 / m / n n(n — 1) 
Using this with (14. 6[) and (14.91) yields 

n n(n — 1) m\ m — 3 m 
This completes the proof of Proposition 14.11 and hence of Theorem 12.11 □ 
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Proof of Theorem \2.S[ Suppose n, m both go to infinity in a linked man- 
ner, in such a way that n/m — > a G (0,oo). Then it can be shown (see 
Theorem II. 1.1 on pages 37-38 of [11]) that EF ~ n(l — e~ a ), and 

Var(F) ~ n (e~ a {l - e~ Q ) + e~ 2a (a(l - «))) = ng(af . 

Substituting these asymptotic expressions into (I2.2p and using the fact that 
in this asymptotic regime, (nrj(n,m)) — > 16 + 24a(2 + 2a), yields (12.41) . 

5 The non-uniform case: proof of Theorem 
[2^1 



For this proof, we use the following notation. Given n, m, and the probability 
distribution p on [m], let Xi,X 2 , ■ ■ ■ ,X n+1 be independent [m]-valued ran- 
dom variables with common probability mass function p. Given i < j < n+1, 
set Xj := (Xl, . . . , Xj) and 

r {X 1 ,X 2 ,...,X i _ 1 ,X i+1 ,...,X j ) if l<i<j 
X iV := I (X 2 ,...,X j ) if i = l 

l (.V; V ; ,) if i=j. 

Given any sequence x = (xi, . . . ,Xk), set 

^(x)=^(i- n (i-i{^=^})], (5.i) 

which is the number of non-isolated entries in the sequence x, so that in 
particular, Y = H(K n ). We shall use the following consequence of Jensen's 
inequality: for all k 6 N, 

(h + t 2 + --- + t k ) 2 < k(tj + ■■■ + %), V (h,...,t k ) Gl fc (5.2) 

We shall also use several times the fact that — t _1 ln(l — t) is increasing on 
t G (0, 1) so that by (12.71) for all x G [m] we have 

ln(l - p x ) > (11 ln(10/ll)K > -lMp x (5.3) 

whereas (1 — e _ *)/t is decreasing on t G (0, oo) so that by (12.51) . for any a > 
and x G [m] we have 

1 - e~ anpx > (1 - e~ a7 )(np x ./7). (5.4) 
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Proof of ( \2.11\i . We use Steele's variant of the Efron-Stein inequality [JJ] . 
This says, among other things, that when (as here) X%, . . . ,X n+1 are inde- 
pendent and identically distributed random variables and if is a symmetric 
function on M. n , 

1 n 

VarH(X u . . . , X n ) < - ^ E [(H(X n ) - H (X {n+m )) 2 } 

i=i 

= (n/2)E [(H(X n ) - #(X (n+1) \ n )) 2 ]. 

Hence, by the case k = 2 of (15. 2p . 

VarF < n(E [(#(X n ) - i^X^)) 2 ] + E [(H(X {n+1) \ n ) - //(X^)) 2 ]) 

= 2nE[(#(X n )-#(X n _ 1 )) 2 ]. 

With Mj defined by (12TT]) . if (X n )— iJ(X n _i) is equal to l{M n > 1} + 1{M„ = 
1}, so is nonnegative and bounded by 21{M n > 1}. Therefore, 

Var[F] < 8nP[M n > 1] < 8nEM„ < 8n 2 53^- □ 

a; 

Proof of ( \2.12\) . Construct a martingale as follows. Let T§ be the trivial 
cr-algebra, and for % <G [n] let ^ := <j{X\, ■ ■ ■ , -Xj) and write Ej for conditional 
expectation given jFj. Define martingale differences Aj = E i+ iY — E jY. Then 
7-17 = ^"Jo 1 anc ^ by orthogonality of martingale differences, 

n— 1 n— 1 

Var[y] = 53^[A 2 ] = ^ Var[Ad. (5.5) 

i=0 i=0 

We look for lower bounds for E[A 2 ]. Note that 

A, =E i+1 [Wi], where W t := F(X n ) - (X (n+1) \ (m) ). (5.6) 

Recall from (12.11) that for i < n, Mj+i denotes the number of balls in the 
sequence of n balls, other than ball i + 1, in the same position as ball i + 1. 
Similarly, define M n+ i and M| (for k G [n + 1]) by 

M„ +1 := 53 = X n+1 }; M* : = 53 1{X, = X k }, (5.7) 

ie[n] ie[i]\{fc} 
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so that M n+ i is the number of balls, in the sequence of n balls, in the same 
location as ball n + 1, while M l k is similar to M*., but defined in terms of the 
first i balls, not the first n balls. 

Set ho(k) := l{k > 1} + l{k = 1}. Then tf(X n ) - H(X nX{i+1) ) = 
h (M i+1 ), andifX n+ i ^ X i+1 then if (X (n+1 )\ {m) )-#(X n \ (m) ) = /i (M n+ i), 
so that Wj = /i (Mj +1 ) — h (M n+ i) in this case. For taking E i+1 -conditional 
expectations, it is convenient to approximate h (M i+ i) and h (M n+1 ) by 
ho(Ml +l ) and ^o(^n+i) respectively. To this end, define 

Zi := W, - (h (Mi +1 ) - MK+i))- (5.8) 

Since ho(M- +1 ) is jF i+1 -measurable, taking conditional expectations yields 

h (Mt +1 ) = E i+1 [Wi] +E i+1 [h (Mi +1 )] - E i+1 [Zi\. (5.9) 

Set 5 := (2887e 1 057 ) -1 . We shall show that for i close to n, in the sense 
that n — 5n < i < n, the variances of the terms on the right of (15. 9p . other 
than Ej +1 [Wi], are small compared to the variance of the left hand side, 
essentially because E i+ i[/i (M^ +1 )] is more smoothed out than /i (M* +1 ), 
while P[Zi ^ 0] is small when i is close to n. These estimates then yield a 
lower bound on the variance of Ej + i[Wj]. 

First consider the left hand side ho(Ml +1 ). This variable takes the value 
when M l i+1 = 0, and takes a value at least 1 when M? +1 > 1. Hence, 

VarM^ +1 )] > (l/2)min(P[JW? +1 = 0],P[M| +1 > 1]). (5.10) 

For i < n, by ([53]) and (l?3j) . 

p\mi +1 = o] = j^p^-p-y ^ J2p^-p^ n 

X X 
X X 

For z > (1 — 8)n we have i > n/2, so by (15.41) and the fact that 7 > 1 by 

P[Ml +l > 1) = Y,Px(l - (1-PxT) > - e " nW2 ) 

> - e-^)np x h > (1 - e" 1 / 2 ) 7 - 1 J>j£. (5.12) 
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Since 7 > 1, and e 105 < 1 — e 5 , the lower bound in (15.111) is always less 
than that in (I5.12p . so combining these two estimates and using (I5.10p yields 

Var[h (M l i+1 )} > (l/2)7~ 1 e - L057 ^ np 2 x , ie[n- 5n, n]. (5.13) 

X 

Now consider the second term E i+1 [/z (M^ +1 )] in the right hand side of (|5.9|) . 
Set N l x := Yi j= i HXj = x}, and for 1 < i < i set M\ to be N l X( - 1. Also 
set h (k) — (k + l)" 1 /^^ + 1). Then, since h (0) = 0, we have that 

i 

Var E i+1 [h (Mi +1 )} = Var^PMK) = Var Y,PxM M D 

X j = l 



^Var[np Xl h (Mi)} + ^ ^ Cov | „,> , V| Ml/; ). ///;, Vj // ( ,(.1/^ 



Suppose i < n. Since < npxi^o(-^i) < 2npxi, (I2.5[) yields 

-^Var[np Xl /i (M*)] < n _1 E [4n 2 pxJ = 4 ™^Px < 4 7$^ 



(5.14) 



while by Lemma [3.51 and (12.51) . 



Cov 



< 



(3 + 3 7 )4 7 2 ^ 



/io (MJ ) , npx 2 h (M l 2 ) 

Combining the last two estimates on (I5.14p and using assumption (12. 8ft yields 

Var(E,[VK+i)D < (1 + 3 7 + 3 7 2 )4 7 

X 

< (lSV- 057 )- 1 ^^; 



,pI- 



(5.15) 



We turn to the third term in the right hand side of (15.91) . As discussed just 
before (I5.8p . when X n+ x 7^ X i+ i we have Wi = h (M i+ i) — /i (M„ + i), and it 
is clear from the definitions (15. 6p and (15.71) that if X n+1 = X i+ i then both 
Wi and h (M- +1 ) — h (M^ +1 ) are zero, and therefore by (15.81) . 



Z, = (h (M i+1 ) - h (Mi +1 ) - h (M n+1 ) + h {M l n+1 ))l{X n+1 + X l+l }. 
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By the conditional Jensen inequality, 

Var(E m [^]) < E[(E m [Z,]) 2 ] <E[Zf\. 

The random variable h (M n+ i) — ho(M^ +1 ) lies in the range [—2,2] and is 
zero unless Xj = X n+ \ for some j G (i, n]. Similarly, h (M i+ i) — h (Ml +1 ) 
lies in [—2,2] and is zero unless Xj = Xi + i for some j G (i + l,n]. Hence, 
using (15.21) and the definition of 5 yields for i G [n — 5n, n] that 

Var(E i+1 [Z i ]) < 2(4P[M n+1 ^ M* +1 ] + AP[M l i+1 ^ M\ +l }) 

< 165nJ2pl < (iSTe 1 - 057 ) -1 ^^" ( 5 - 16 ) 

X X 

By O and the case k = 3 of 
Var[/ i0 (M| fl )] < 3(Var(E i+1 [Wd) +Var(E i+1 [/io(M; +1 )]) + Var(E i+1 [Z i ])). 
Rearranging this and using (15.131) . (I5.15p . and (15.161) yields the lower bound 



/l 9 \ P" 1057 p-1-057 _ 

v«r(R W [ W a ) >(i-l)i^-E'ri = V? 



for i G [n — 5n, n]. Since the definition of 5, the condition (12.81) on n and the 
assumption 7 > 1 guarantee that n5 > 2, and since > 2£/3 for t > 2, by 
(15.51) and (15. 6p we have 



Var[F] > [Sn^l^e^Y^^pl > (5n)(27-fe im ^)~ 1 nJ2pl 

X X 

which is (12TT2D . □ 



6 Proof of Theorem 12.3 



Proof of (HJ| . Write a for oy, and for t G R set F{t) := P[(Y — EY)/a < t]. 
Set z : = o- _1 (LEFj - EY), and set := z + (1 -e)/a, for some e G (0, 1). 
Then since Y is integer- valued, F(z{) = F{zq). On the other hand, by the 
unimodality of the normal density, 

Hzx) - Hz ) > (1 - e)a-\2n)- 1 / 2 exp(-l/(2a 2 )) 
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so that Dy is at least half the expression above. Making e j and using the 
fact that e _1 /( 2cr ' > e -1 / 2 for cr > 1, gives us (12. 6p in the case where a > 1. 

When cr < 1, we can take z 2 < < 23, with z 3 = ,2 2 + l and ^(23) = -^(^2)- 
By the 68 — 95 — 99.7 rule for the normal distribution, $(23) — $(^2) > 1/3, 



So in Theorem 12.31 the difficulty lies entirely in proving the upper bound 
in (12.91) . under assumptions (12. 7p and (12. 8p which we assume to be in force 
throughout the sequel. By (12. 8p we always have n > 1661. 

As before, set ho(k) := l{k > 1} + l{k = 1}. Define for nonnegative 
integer k the functions 



The function h\{k) may be interpreted as the increment in the number of 
non-isolated balls should a ball in an urn containing k other balls be removed 
from that urn with the removed ball then deemed to be non-isolated itself. If 
q = then the ball removed becomes non-isolated so the increment is 1, while 
if q = 1 then the other ball in the urn becomes isolated so the increment is 
-1. 

The function h 2 (k) is chosen so that h 2 (k) + 2hi(k) (for k > 1) is the 
increment in the number of non-isolated balls if two balls should be removed 
from an urn containing k — I other balls, with both removed balls deemed to 
be non-isolated. The interpretation of is given later. 

We shall need some further functions hi which we define here to avoid 
disrupting the argument later on. For x G [m] and k G {0} U [n — 1], let 
7Tk{x) be given by the tt^ of (13.31) when v — n — 1 and p = p x . With the 
convention • H-i(x) := • hi{— 1) := 0, define 



so Dy > 1/6, giving us (12.61) in the case where a < 1. 



□ 



h x {k) := 1 - h (k) = l{k = 0} - l{k = 1}; 
h 2 (k) := 21{k = 1} - l{k = 2}; h 3 {k) := l{k = 1}. 




h 5 (k, x) := n k (x)/ (n - 1), h 6 (k) := kh 2 (k); 




kh 5 (k, x)(k - l)h 2 (k - 1) 



n 



For i = 0, 1, 2 



3,6 define hi(k,x) := /ij(fc). For each z define 
fci(A;,a;) := ^(A; + l,x)/(k + 1). 



(6.1) 
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Sometimes we shall write hi(k) for hi(k,x) when i G {0,1,2,3,6}. Define 
\\hi\\ := sup fea . \hi(k, x)\ and \\hi\\ := sup fca , \hi(k, x)\. 

Now we estimate some of the hi functions. Since 710(2) = 1 we have 
/14(C), x) = hj(0,x) = for all x, which we use later. Also, by Lemma [3.31 

-1 < hi(k,x) < (6.2) 

and h 7 (l, x) = 1 + h 4 (l, x)(l —n^ 1 ) — 2/n so that — 1/n < h 7 (l, x) < 1. Also, 
since (12. 8p implies n > 1661, 

h 7 {2,x) = h 4 (2,x)(l + 2U- 1 ) - A **W + — G [-1,4/n]. 

n(n — 1) n 

Also, h 3 {3) = hx(2) = so that by flgSP 

M3,x)= G ^- + h 4 (3,x)e[-i,i], 

n{n — 1) 

again since n > 1661. For k > 4, h 7 (k,x) = h 4 (k,x) G [—1,0]. Thus, 

IIM ^ Il^ll < 1; (6.3) 
||M=2; \\h\\ < (n - 1) _1 ; ||/ib||=2. (6.4) 

The strategy to prove Theorem 12.31 is similar to the one used in the 
uniform case, but the construction of a random variable with the distribution 
of Y', where Y' is defined to have the Y size biased distribution, is more 
complicated. As in the earlier case, by Lemma 13.21 if / is uniform over [n] 
then the distribution of the sum Y conditional on Mj > has the distribution 
of Y' . However, in the non-uniform case the conditional information that 
Mi > affects the distribution of Xj. Indeed, for each i, by Bayes' theorem 

P[ Xl = x\ Ml >0} = J xi \~ ( \~ PxT ~X =■ P- ( 6 - 5 ) 
z2yPy{ 1 ~ I 1 -Py) n l ) 

Therefore the conditional distribution of (X\, . . . , X n ), given that Mi > 0, is 
obtained by sampling Xj with probability mass function p and then sampling 
{Xj, j G independently with probability mass function p, conditional 

on at least one of them taking the same value as Aj. Equivalently, sample the 
value of Xj, then Mj according to the binomial Bin(n — l,pxj distribution 
conditioned to be at least one, then select a subset J of [n] \ {i} uniformly at 
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random from sets of size M i: let the values of Xj,j G J be equal to X i: and 
let the values of Xj,j ^ J be independently sampled from the distribution 
with the probability mass function of X given that X ^ X{. 

Thus a random variable Y", coupled to Y and with the same distribution 
as Y', can be obtained as follows. First sample X\, . . . ,X n independently 
from the original distribution p, and set X = (X±, ...,X n ); then select / 
uniformly at random from [n]. Then sample a further random variable X 
with the probability mass function p. Next, change the value of Xj to that 
of Xq; next let N denote the number of other values Xj,j G [n] \ {1} which 
are equal to X , and let nk = iik{Xo) be defined by (13.31) with v = n — 1 and 
p = px - Next, sample a Bernoulli random variable B with parameter ir^, 
and if B = 1 change the value of one of the Xj,j G [n] \ {1} (J = J, with J 
sampled uniformly at random from all possibilities) to Xq. Finally, having 
made these changes, define Y" in the same manner as Y in the original 
sum (12.11) but in terms of the changed variables. Then Y" has the same 
distribution as Y' by a similar argument to that given around (14. ip in the 
uniform case. 

Having defined coupled variables Y, Y" such that Y" has the Y size biased 
distribution, we wish to use Lemma 13.11 To this end, we need to estimate 
the quantities denoted B and A in that lemma. The following lemma makes 
a start. Let Q be the cr-algebra generated by the value of X, and for x G [m] 
let N x := Y^i=i = x } be the number of balls in urn x. 

Lemma 6.1. It is the case that 

\Y"-Y\<3, a.s. (6.6) 

and 

E [Y" - Y\g] = 2 + (j2p x h 5 (N x , xjn- 1 ^ h^M, 

\ x i=l 



n 

i=l J i=l 




Proof. We have 

E \Y" - Y\G] = E [Y" - Y\X] = ^2p x E x \Y" - Y\X], (6.8) 
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where E X [-|X] is conditional expectation given the value of X and given also 
that X Q = x. The formula for E X [Y" — V|X] will depend on x through the 
value of N x and through the value of p x . 

We distinguish between the cases where I is selected with Xi = x (Case 
I) and where / is selected with Xj ^ x (Case II). If N x = k, then in Case I 
the value of N on which is based the probability tt^(x) of importing a further 
ball to x is k — 1 whereas in Case II this value of N is k. The probability of 
Case I occurring is k/n. 

The increment Y" — Y gets a contribution of h\{Mi) from the mov- 
ing of Ball i to x in Case II, and gets a further contribution of h\(Mj) + 
/i2(Mj)l{Xj = Xj} if Xj is also imported to x from a location distinct from 
x. Finally, if N x = k the increment gets a further contribution of h^{k) from 
the fact that if there is originally a single ball at x, then this ball will no 
longer be isolated after importing at least one of balls / and J to x (note that 
tto(^) = 1 so we never end up with an isolated ball at x). Combining these 
contributions, we have (16.61) . and also that for values of X,x with N x = k, 

E x [Y"-Y\X} = h 3 (k) + k J k n 1 }^ MM^ + n- 1 MM,) 



E (/ii(Mf) + h 2 (Mi)l{Xi = Xj}) 



n(n — 1) 

where in the right hand side, the first sum comes from Case I and the other 
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two sums come from Case II. Hence, if N x = k then 



E x [Y"-Y\X] = h 3 (k)+ ( . 7rfc(rg) 1 . V Mih 2 (Mi) 

1 n n — 1 ' 



\n(n— 1) n n(n — 1J / ' 

= v*) + 2+MM) ( (E w)) - kh(k - 1) 



.i=i 

n 



,i=l 

n \ / _ n 



2 + MA,*) - ( /io(M 4 ) I - [ 



n 14 — ' / \ n 

i=i / \ i=i 



n 

i=i 



Then by (EH]) we have (Q- □ 

The next lemma is based on the observation that since /14(C), x) = /17(C), x) = 
0, the sums of the form Ex in ( 16. 7j) are over non-empty urns so can be ex- 
pressed as sums over the balls, i.e. of the form EiLr We need further no- 
tation. Set & := pXih^MijXi), and 1} := h (Mj). Set 6 := E [Tj] (this does 
not depend on j), and Tj := Tj — b. Again write E(^>^ for E!U E,- 6 [n]\w 

Lemma 6.2. is i/ie case i/iai 

Var(E [y" - y|y]) < 12 (n - l) -2 + 3n" 2 Var [ ^ ^f 3 

+3VarJ]([/i 7 (M i ,X i ) - (1 - n- l )bh A {M h Xi)]p Xt 

-[n' 1 h (M l )(2 + pxMMuXi))]). (6.9) 
Proof. As in Section HI ( 14.21) holds here too. So it suffices to prove ( 16.91) 
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with the left hand side replaced by Var(E [Y" — Y\Q]). Set 

n 

p(X) :=2 + J2PMN x ,x)n- 1 J2 h z(M i ). (6.10) 

x i=l 

Using (j6.ip . we reformulate the sums in (16. 7p as follows. Since h 7 (0,x) = 

0, 

J2PMN*,x) = X> ( k7 ^M + +l) l) ) = EPxMM^X,). (6.11) 
Similarly, J2 X P^( N ^^ X ) = Yn=iPxM{Mi, X { ) so that 



n in 



/^sPxh^Ng, x) J n ^^(M,) = n- x Y^$xM M i, x i)W M i) 

i=i \ i=i , 

+n" x E pxMMiiXjhoiMj). (6.12) 



Substituting ([OOP . flBTTTj) and fl6TT2l) into dUTJ) gives 

\i=i J \i=i n , 

+p(X.)-n- 1 PxMMiiXjhoiMj). (6.13) 

The last sum in (16. 13ft can be rewritten as follows: 



n 



E W= E (%+6(2i-6)) = (n-l) E 6 & + E 

Substituting into (16.131) yields 



E [Y" - Y\Q] = p(X) - In" 1 E ^ + E^^ M - X *) 
-(l-n-^bpxMMuXi) - K 1 /i (M i )(2 + p x l 4 (M J ,X i ))]). 
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Bv dSJSl) and (IQjl . |p(X)-2| < 2(n- l)" 1 , so that Var(p(X)) <4(n-l)- 2 . 

By f!5.2p . we then have (16.91) as asserted. □ 

Now we estimate p x . By (15.31) . (1 — Py) 11 ^ 1 > e~ 1 - 05nPy for y E [m], so 

1 - (1 - p y ) n - 1 < 1 - e - lmnp * < 1.05np v , (6.14) 

and by (15.41) . (12.51) and the assumption that n > 1661 by (12.81) . 

1 - (1 - p y ) n ~ l > 1 - e -°- 9np y > (1 - e'^npyh > (1 - e - a9 )n^/ 7 

> (0.55)np y /7. (6.15) 

By (E3D, (1Q41) and (jOSli . for all x G [m] we have that 

ft < (6.16) 
By (J23]) and (1231"]) . we have further that 

||p|| := supfe) < (27) sup < —^- 2 (6.17) 

m x Z^y a Py 11 Z^iy Py 

Also, by (16U6D . (1231) and ([2TLj) . 

Efe < < (6-19) 



y n 



Ef) 2 < ^fE^ < ^fE £ A 32 ^ ( 6 20 ) 

PXl " (E,^) 2 " (VarF)E^ 2 " nVary ^ 2Uj 

Lemma 6.3. With C(j) given by it is the case that 

Var(E [Y" - Y\Y}) < (< P/ 7 |T - (6.21) 

Vary 

Proof. We shall use the fact that by ( 12. lip VarF < 8^7 so 

n- 1 < 87(VarF)" 1 . (6.22) 
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We estimate in turn the two variances in the right hand side of (16.91) . First 
consider the single sum. Let 5* denote the ith term in that sum, i.e. set 

S, := [h 7 (Mi, Xi) - (l-n-^bhtiMuXjtfxi 
-hoiM^+pxMMuX^n- 1 , 

and set 5$ := 5^ — E 5$. By (16.21) and (I6.3p . along with the fact that h (k) e 
[0, 2] so < b < 2, the coefficient of px v in the definition of Si, lies in the 
range [—1,3], while the coefficient of n" 1 lies in the range [—4,0]. Hence, 
\3i + 1 1 < 3p Xi + I- By (Q, (E2QD and <$M, 

3nVai [Si] < 3nE [(Si + (2/n)) 2 ] < 6n (9E [p^J + 4n~ 2 ) 

< 1728 ^ + 192 T-. (6.23) 
VarF 1 ; 

Also, in the notation of Lemma 13.51 if we write Si = ip{Xi, Mi) we have 
rng(-0) < 4||p|| + 4n _1 , and also H^H < 4||p|| + An^ 1 . Hence, by ( 16. 17)) and 
(!6TT5|) . followed by fl2Tll and then fl23|) . 



max(rng(V>), ||^||) < 4 + 



'VarF^pM / 3 2 7 e 



2 7 6 ) \l n 2 VavYJ2 y Pl 



7 3 J V n 2 VarY ^2 y Py \ 7 2 / y n 2 VaxY YlyPy 
By Lemma [3.51 

3n 2 Cov [Si, S 2 ] = 3n 2 E < 9(1 + 7) (32 7 6 /Varr) (16 + 16 7 ~ 2 + 4 7 ~ 4 ) 

4608( 7 7 + 7 6 + 7 5 + 7 4 ) + 1152( 7 3 + 7 2 ) 
- VarF ' 

Combining this with H6.23j) yields 

n 

3Var Si = 3nVar[Si] + 3n(n — l)Cov[Si, S2] 

< C^p) (47 7 7 + 47 7 6 + 64 7 5 + 47 7 4 + 12 7 3 + 12 7 ^.24) 



i=l 
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Consider now the double sum in (16.91) . Writing (n)k for n\/(n — k)\, we have 
Var ^^(n^Cov^,^ 

+(n) 3 (Cov(£iT 2 ,£iT 3 ) + CovfeTx, £ 3 7\) + 2Cov(£ 1 T 2 , ^Je^) 

For the first term of the right hand side of (I6.25p . observe that 

Cov(6T 2 , £ 3 T 4 ) = E l^fi&f,} - E [£iT 2 ]E [£ 3 T 4 ] < E {^f 2 ^f 4 }, 

and that > & > -p Xi by AO]) , while < Tj < 2. So by Lemma EH flSTTTl) 
and (KTTgj) . 

3n 2 Cov(6T 2 ,£ 3 T 4 ) < 12rz 2 1| p || 2 (9 + 9 7 ) ^ 108(1+7) 

= (34567 6 + 34567 7 )/Vary. (6.26) 
Now consider the last term in fl6.25p . By (16.201) . 

3nCov(6?2, < 3nE [£T 2 T 3 ] < 12nE^ < (6.27) 

Varr 

while by fl6U9]) and fl6T22|) . 

37i(Cov(e 2 f 1 , £ 3 7\) + 2Cov(£ 1 f 2 , e 3 f0) < 3nET 2 £ 2 £ 3 + QnE^f^ 

1152'y 5 

< 36nE [p Xl Px 2 ] < 144 7 4 rT 1 < (6.28) 

The middle term in the right side of (16.251) is smaller; since E [px 1 Px 2 \ — 
E[&J, « gives 



3(Var(eiT 2 ) + Cov^T^)) < 24E [p 2 ] < 



XlJ - nVarY' 



and since n > 1661 by (J211, combined with (RT25]I . flSIZZD, and flQgJ , 

this shows that 



(357 7 + 357 6 + (15.9) 7 5 ). (6.29) 
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Also, by (16.221) we obtain 

_ 2 < (J2_\ f n \ _! < / 96 x 1661 N 7 < 7 
^~ ^ - V1660 / \n-l) n ~ V 1660 2 / VarF ~ lOVarF' 

Combining this with (jO]) . fl6T2l) and fl6T29|) yields 

Var(E [Y" - < (^7) (82( 7 7 + 7 6 ) + 8O7 5 + 47 7 4 + 12( 7 3 + l 2 )). □ 



Proof of Theorem \2.3[ It remains to prove (12.91) . By ( 16. 14ft . 
Ey = n^p,(l-(l-p i r 1 ) < lMn 2 J2pl 

X X 

so that by (12TT2D . 

/iy/a^ < 81657 2 e 2 - 17 . (6.30) 

We can apply Lemma EH] with B = 3 by (16.61) . According to that lemma, for 
(13.11) we need B < a 3 / 2 / 'y/6jl, i.e. cr 3 > 54/x. By (I6.30p . a sufficient condition 
for this is that a > 54(8165)7 2 e 217 . But if this condition fails then since 
577 > 54, the right hand side of (I2.9P is greater than 1 so we are anyway 
guaranteed that (I2.9P holds. 

Therefore, from now on we may assume that cry > 54(8165)7 2 e 2 ' 17 , so 
that (13.11) is valid. Using (I6.30P and Lemma [6.31 this gives 

D Y < ay 1 (1.2 + 81657 2 e 2 ' l7 (576 + lOSc^ 1 + 23(7)) 

with C — C(j). We may then deduce (1^9]) . □ 
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